Lecture 8 : Shannon ’ s Noise Models

In the figure above, source coding and channel coding are coupled. However, Shannon’s source coding theorem allows us to decouple both these parts of the communication and study each of these parts separately. Intuitively, this makes sense: if one can have reliable communication over the channel using channel coding, then for the source coding the channel effectively has no noise. For source coding, Shannon proved a theorem that precisely calculated the amount by which the message can be compressed: this amount is related to the “entropy” of the message. We will however, not talk about source coding in any detail in this course. From now on, we will exclusively focus on the channel coding part of the communication setup. Note that one aspect of channel coding is how we model the channel noise. We have seen Hamming’s worst case noise model in some detail. Next, we will study some specific stochastic channels.